Robust Combination of Model Selection Methods for Prediction

One important goal of regression analysis is prediction. In recent years, the idea of combining different statistical methods has attracted an increasing attention. In this work, we propose a method, l1-ARM (adaptive regression by mixing), to robustly combine model selection methods that performs well adaptively. In numerical work, we consider the LASSO, SCAD, and adaptive LASSO in representative scenarios, as well as in cases of randomly generated models. The l1-ARM automatically performs like the best among them and consequently provides a better estimation/prediction in an overall sense, especially when outliers are likely to occur.